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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
7/31/2008 has been entered. 

Specification 

1 . The disclosure is objected to because of the following informalities: Claims 1 2-1 9 
recite a 'computer readable storage medium'. It is noted that the applicant's 
specification, in paragraph 43 indicates a computer readable medium and definitions, 
and further discloses a storage medium. However there is no explicit disclosure or 
definition of a computer readable storage medium sufficient for one to differentiate the 
computer readable storage medium, from the described computer readable medium. 
See 37C.F.R. 1.75(1)(d). 

Appropriate correction is required. 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 
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2. Claims 12-17 and 19 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. The claims recite a "computer 
readable storage medium". However, the specification indicates that such a medium 
can comprise a carrier wave (Spec. par. 43). Therefore, the invention can correctly be 
interpreted as an electromagnetic signal which is non-statutory. See MPEP 2106.01 . 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

Claims 1-6, 8-17, 19-23 and 25-31 are rejected under 35 U.S.C. 112, first 
paragraph, as failing to comply with the enablement requirement. The claim(s) contains 
subject matter which was not described in the specification in such a way as to enable 
one skilled in the art to which it pertains, or with which it is most nearly connected, to 
make and/or use the invention. 

The claims recites a motion estimation method generally applicable to multi-view 
video that constrains the search range associated with a second frame to an area 
relative to the position of an epipolar line in the second frame which corresponds to one 
or more pixels in the first frame. The claim further discloses that the search range is 
defined by a desired correlation between efficient compression and semantic accuracy. 
The claim also states that semantic accuracy relies on the use of geometric 
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configurations of cameras capturing the multi-view video sequence. The applicants 
specification indicates that a semantically accurate match reflects the 'true' motion in the 
video sequence rather than simply a good match for efficient compression (Spec. pars. 
5 and 23). The applicants claims and specification thus assert that such semantic 
accuracy is obtained by matching portions of a current frame to portions of a previous 
frame by searching in an area near an epipolar line in that frame. The examiner does 
not believe that the applicants claims enable one of ordinary skill in the art to make or 
use the invention. 

The MPEP indicates that a rejection based on enablement should begin by 
providing a basis on which to question the enablement provided by the applicant's 
specification. See MPEP 2164.04. First, the applicants claims do not enable use of the 
invention to select a desired trade-off between efficient compression and semantic 
accuracy. In a multi-view sequence of known geometry the image of a three 
dimensional object captured from one view is constrained to lie upon an epipolar line in 
another image. This relationship is commonly used as a constraint for determining a 
disparity between two images taken at the same instant in time. However, if the three 
dimensional object moves it is no longer constrained to lie upon the same epipolar line 
(See Multiple View Geometry in Computer Vision (hereafter Hartley), Fig. 8.2(b) on 
page 220 showing different epipolar lines as object moves). The applicants invention 
asserts that 'true' matches for an object in a first frame can be obtained by searching 
near or along an epipolar line at a corresponding position in a second frame captured at 
a different instance in time, which will hereafter be referred to as the corresponding 
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epipolar line. As shown in Hartley Fig. 8.2 if an object has undergone motion in the time 
between the capturing of the two frames the epipolar line determined in the second 
frame will be a different epipolar line than the object may be found upon within that 
second frame. The epipolar line upon which the object can actually be found in the 
second frame will hereafter be referred to as the object epipolar line. An object in an 
image is not necessarily constrained by a particular epipolar line over time. Therefore a 
search along the corresponding epipolar line is not equivalent to a search along the 
object epipolar line, and does not necessarily have any affect on finding a 'true' or 
semantically accurate motion. Based upon the understanding above, the examiner does 
not believe that the applicants invention, without further guidance, explicitly enables a 
selection between semantic accuracy and efficient compression. 

Having established a reason for doubting the accuracy of the applicants 
specification the examiner will now turn to determining if such inaccuracies would 
require undue experimentation. The MPEP indicates that the factors to be considered in 
determining undue experimentation may include the breadth of the claims and the 
amount of direction provided by the inventor. 

The claims relate to multi-view video encoding, but are claimed to apply generally 
to all types of motion in the encoding of multi-view video. The evidence listed above 
casts doubt as to whether the invention as described in the specification will function for 
general motion. The invention may have applications when the motion occurs from 
known camera motions where epipolar constraints can be derived (See Hartley 8.3 
pages 228-231). However, the applicant does not mention camera motion, or given any 
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direction to deriving epipolar constraints from camera motion. Therefore, the examiner 
believes that an undue amount of experimentation would be required to determine 
where the invention would be applicable. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-3, 12-14, 20-22 and 26-28 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Sohn et al (20030202592) in view Chen (6043838). 

In regard to claim 1 and 2 Sohn discloses a multi-view video encoding method 
and apparatus that includes: 

identifying one or more pixels in a first frame of a multi-view video sequence 
(Sohn Fig. 2 and par. 12 note block based motion/disparity prediction will select a block 
of pixels for prediction); 

constraining a search arrange associated with a second frame of the multi-view 
video sequence the second frame offset in time from the first frame (Sohn. Fig. 3 and 
pars. 11-16 note motion/disparity prediction for right pictures from left pictures offset in 
time, further note par. 52 setting constraints on disparity searches may be set as 
desired); and 
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searching the second frame within the constrained search range for a match of 
the one or more pixels identified in the first frame for subsequent use in computing a 
motion vector for the one or more pixels (Sohn pars. 11-16 note disparity prediction and 
disparity vectors). 

It is noted that Sohn does not disclose particular details relating to 
motion/disparity estimation. However, Chen discloses a multi-view video encoding 
method in which a search range in a second frame is constrained relative to the 
position of the one or more pixels in the first frame, and further discloses finding a 
position of an initial seed using a disparity vector (Chen Fig. 9 and col. 7 line 38 to col. 8 
line 18 note determining an initial seed for horizontal displacement, specifically col. 8 
lines 12-18 initial seed may have a value between 5 and 15 pixels offset, further note 
col. 10 lines 36-59 note disparity estimation compares macroblocks in a 24x24 search 
area around the seed, in the given example the search area is constrained to include 
second frame pixels corresponding to the selected pixels in the first frame). It is 
therefore considered obvious that one of ordinary skill in the art at the time of the 
invention would recognize the advantage of including motion/disparity prediction as 
taught by Chen in the motion/disparity estimation of Sohn in order to improve coding 
efficiency in multi-view video as suggested by Chen (Chen col. 3 lines 61-64). 

It is further noted that neither Sohn nor Chen explicitly disclose constraining a 
search area relative to the epipolar line. However, both Sohn and Chen relate to multi- 
view video therefore for any given pixel in an image there is inherently an associated 
epipolar line that describes constraints between the image and a concurrent image from 
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another viewpoint. Chen further discloses that pixels in the second frame corresponding 
to the identified pixels in the first frame may be included in the constrained search area. 
Therefore, Sohn in view of Chen discloses constraining a search range associated with 
a second frame of a multi view sequence to an area relative to a position of an epipolar 
line in the second frame because at least a portion of the epipolar line is within the 
search range. 

It is further noted that neither Sohn nor Chen expressly disclose that the 
constrained search area is defined by a desired correlation between efficient coding 
and semantic accuracy. However, any constrained disparity/motion search inherently 
represents some desired correlation between efficient coding and semantic accuracy. 
A less constrained will inherently improve coding efficiency by increasing the number 
of possible matches and thereby improving compression, whereas a constrained or 
smaller search is likely to improve semantic accuracy because most frame to frame 
motion in a video sequence has a small magnitude. Therefore any search area 
inherently define some correlation between efficient coding and semantic accuracy. 

In regard to claim 2 refer to the statements made in regard to claim 1 above. 
Further note that the position of an epipolar line inherently depends on the geometric 
configurations of cameras. 

In regard to claim 3 refer to the statements made in the rejection of claim 1 
above. Sohn and Chen both disclose blocks of pixels (Sohn par. 12, Chen col. 10 
lines 48-58) 

In regard to claims 12-14, 20-22 and 26-28 refer to the statements made in the 
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rejection of claims 1-3 above. 

4. Claims 4-6, 15-17 23, and 30 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Sohn et al (20030202592) in view Chen (6043838) and in further 
view of Van Horn (6243599). 

In regard to claims 4 and 5 refer to the statements made in the rejection of claim 
1 above. It is noted that neither Sohn nor Chen disclose details of calculating epipolar 
lines, however Van Horn discloses a method of calculating epipolar lines in a multi-view 
video system using a fundamental matrix (Van Horn col. 9 line 36 to col. 10 line 38). It is 
therefore considered obvious that one of ordinary skill in the art would recognize the 
advantage of including epipolar line computations as taught by Van Horn in the 
invention of Sohn in view of Chen in order to reduce prediction processing for 
concurrent disparity estimations as suggested by Van Horn (Van Horn col. 10 lines 3- 
15). 

In regard to claim 6 refer to the statements made in the rejection of claim 1 
above. Chen further discloses determining the parameters of a window covering the 
initial seed and the epipolar line based on the desired correlation between efficient 
compression and semantic accuracy (Chen col. 10 lines 48-58 note determining center 
point (x,0) for the search window) 

In regard to claims 15-17 23, and 30, refer to the rejection of claims 4-6, 12, 20 
and 26 above. 
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5. Claims 8-11, 19, 25 and 31 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Sohn et al (20030202592) in view Chen (6043838) and Newman et 
al (6154600) and in further view of Van Horn (6243599). 

In regard to claims 8-1 1 refer to the statements made in the rejection of claim 1 
above. It is noted that neither Sohn nor Chen disclose specifics details regarding user 
input. However Newman discloses a video editing system in which a user can input 
parameters through a slider in a user interface (Newman col. 14 lines 59-64). It is 
therefore considered obvious that one of ordinary skill in the art at the time of the 
invention would recognize the advantage of including user input through a slider 
window in a user interface as taught by Newman in the invention of Sohn in view of 
Carlbom in order to allow closer user control over the process. 

In regard to claims 1 9, 25 and 31 , refer to the rejection of claims 8-11,1 2, 20 
and 26 above. 

Response to Arguments 

Applicant's arguments with respect to claims 1-6, 8-17, 19-23, and 25-31 have 
been considered but are moot in view of the new ground(s) of rejection. 

In response to the applicants arguments made in regard to the objection to the 
specification, and rejection of claims 1 2-1 9 under 35 U.S.C. 1 01 , the applicant asserts 
that because a storage medium has been held to be patentable by the federal circuit 
and a carrier wave is not patentable, a claim to a computer readable storage medium 
cannot be interpreted as carrier wave and is thus patentable. The examiner must 
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disagree. The examiner notes that a properly claimed and supported storage medium is 
patentable subject matter. The issue in the instant case lies in support. The specification 
does not explicitly provide for a computer readable storage medium, rather the 
specification describes computer readable media and indicates that such media can be 
a storage device or a carrier wave. Because the specification does not explicitly define 
'a computer readable storage medium' there is insufficient basis to differentiate between 
the computer readable medium, which includes carrier waves, and the 'computer 
readable storage medium' which the examiner presumes is intended to cover only 
storage devices. To alleviate the issue the examiner would recommend reverting to the 
claim language to a 'computer readable medium' and in the next response specifically 
pointing out which types of computer readable media are claimed and disclaiming all 
others. 



Conclusion 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JEREMAIAH C. HUBER whose telephone number is 
(571)272-5248. The examiner can normally be reached on Mon-Fri 8:00 a.m. - 4:30 
p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571)272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Jeremiah C Huber 

Examiner 

Art Unit 2621 

/Jeremiah C Huber/ 
Examiner, Art Unit 2621 

/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 



